Real Datasets Research Articles

Sub-One Quasi-Norm-Based k-Means Clustering Algorithm and Analyses

Recognizing the pivotal role of choosing an appropriate distance metric in designing the clustering algorithm, our focus is on innovating the k-means method by redefining the distance metric in its distortion. In this study, we introduce a novel k-means clustering algorithm utilizing a distance metric derived from the ℓp\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _p$$\\end{document} quasi-norm with p∈(0,1)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$p\\in (0,1)$$\\end{document}. Through an illustrative example, we showcase the advantageous properties of the proposed distance metric compared to commonly used alternatives for revealing natural groupings in data. Subsequently, we present a novel k-means type heuristic by integrating this sub-one quasi-norm-based distance, offer a step-by-step iterative relocation scheme, and prove the convergence to the Kuhn-Tucker point. Finally, we empirically validate the effectiveness of our clustering method through experiments on synthetic and real-life datasets, both in their original form and with additional noise introduced. We also investigate the performance of the proposed method as a subroutine in a deep learning clustering algorithm. Our results demonstrate the efficacy of the proposed k-means algorithm in capturing distinctive patterns exhibited by certain data types.

Quantile-based cumulative Kullback-Leibler divergence in past lifetime: Some properties and applications

This article introduces a quantile-based cumulative Kullback-Leibler (KL) divergence measure for past lifetimes (QPCKL divergence) and studies its various properties. Unlike the cumulative KL divergence using the distribution function, the proposed measure offers an alternative method for computing the divergence between two random variables, especially when one lacks a closed-form distribution function but possesses a closed-form quantile function. A characterization property of a distribution is derived using the constant QPCKL divergence model. Subsequently, a comparison of some important reliability characteristics using the proposed measure is discussed. The QPCKL divergence for a general family of transformation models is derived. We also propose a non parametric estimator for the measure, and simulation studies are carried out for validation. Finally, the importance of the estimator is also illustrated through real data sets useful in survival studies.

Exploring facial cues: automated deception detection using artificial intelligence

AbstractDeception detection is an interdisciplinary field attracting researchers from psychology, criminology, computer science, and economics. Automated deception detection presents unique challenges compared to traditional polygraph tests, but also offers novel economic applications. In this spirit, we propose an approach combining deep learning with discriminative models for deception detection. Therefore, we train CNNs for the facial modalities of gaze, head pose, and facial expressions, allowing us to compute facial cues. Due to the very limited availability of training data for deception, we utilize early fusion on the CNN outputs to perform deception classification. We evaluate our approach on five datasets, including four well-known publicly available datasets and a new economically motivated rolling dice experiment. Results reveal performance differences among modalities, with facial expressions outperforming gaze and head pose overall. Combining multiple modalities and feature selection consistently enhances detection performance. The observed variations in expressed features across datasets with different contexts affirm the importance of scenario-specific training data for effective deception detection, further indicating the influence of context on deceptive behavior. Cross-dataset experiments reinforce these findings. Notably, low-stake datasets, including the rolling dice Experiment, present more challenges for deception detection compared to the high-stake Real-Life trials dataset. Nevertheless, various evaluation measures show deception detection performance surpassing chance levels. Our proposed approach and comprehensive evaluation highlight the challenges and potential of automating deception detection from facial cues, offering promise for future research.

Beta transformation of the Exponential-Gaussian distribution with its properties and applications

This study introduces a five-parameter continuous probability model named the Beta-Exponential-Gaussian distribution by extending the three-parameter Exponential-Gaussian distribution with the beta transformation method. The basic properties of the new distribution, including reliability measure, hazard function, survival function, moment, skewness, kurtosis, order statistics, and asymptotic behavior, are established. Using the acceptance-rejection algorithm, simulation studies are conducted. The new model is fitted to the simulated and real data sets, and its performance is reported. The Beta-Exponential-Gaussian distribution is found to be more flexible and has better performance in many aspects. It is suggested that the new distribution would be used in modeling data having skewness and bimodal distribution.

The all-pairs vitality-maximization (VIMAX) problem

AbstractTraditional network interdiction problems focus on removing vertices or edges from a network so as to disconnect or lengthen paths in the network; network diversion problems seek to remove vertices or edges to reroute flow through a designated critical vertex or edge. We introduce the all-pairs vitality maximization problem (VIMAX), in which vertex deletion attempts to maximize the amount of flow passing through a critical vertex, measured as the all-pairs vitality of the vertex. The assumption in this problem is that in a network for which the structure is known but the physical locations of vertices may not be known (e.g., a social network), locating a person or asset of interest might require the ability to detect a sufficient amount of flow (e.g., communications or financial transactions) passing through the corresponding vertex in the network. We formulate VIMAX as a mixed integer program, and show that it is NP-Hard. We compare the performance of the MIP and a simulated annealing heuristic on both real and simulated data sets and highlight the potential increase in vitality of key vertices that can be attained by subset removal. We also present graph theoretic results that can be used to narrow the set of vertices to consider for removal.

DAE-GAN: Underwater Image Super-Resolution Based on Symmetric Degradation Attention Enhanced Generative Adversarial Network

Underwater images often exhibit detail blurring and color distortion due to light scattering, impurities, and other influences, obscuring essential textures and details. This presents a challenge for existing super-resolution techniques in identifying and extracting effective features, making high-quality reconstruction difficult. This research aims to innovate underwater image super-resolution technology to tackle this challenge. Initially, an underwater image degradation model was created by integrating random subsampling, Gaussian blur, mixed noise, and suspended particle simulation to generate a highly realistic synthetic dataset, thereby training the network to adapt to various degradation factors. Subsequently, to enhance the network’s capability to extract key features, improvements were made based on the symmetrically structured blind super-resolution generative adversarial network (BSRGAN) model architecture. An attention mechanism based on energy functions was introduced within the generator to assess the importance of each pixel, and a weighted fusion strategy of adversarial loss, reconstruction loss, and perceptual loss was utilized to improve the quality of image reconstruction. Experimental results demonstrated that the proposed method achieved significant improvements in peak signal-to-noise ratio (PSNR) and underwater image quality measure (UIQM) by 0.85 dB and 0.19, respectively, significantly enhancing the visual perception quality and indicating its feasibility in super-resolution applications.

Almost Unbiased Estimators for Population Coefficient of Variation Using Auxiliary Information

The objective of the paper is to propose an almost unbiased ratio estimator for the finite coefficient of variation (CV). In this paper, we have proposed an exponential ratio type and log ratio type estimators for estimating population coefficient of variation. Two real data sets and one simulation study is carried out in support of the theoretical results. Mean squared error and Percent relative efficiency criteria is used to assess the performance of the estimators. It has been shown that the proposed class of estimators are almost unbiased up to the first order of approximation. Also proposed estimators are better in efficiency to other estimators consider in this study.

CICIoV2024: Advancing realistic IDS approaches against DoS and spoofing attack in IoV CAN bus

Considering the complexity of network traffic in IoV operations, methods that can identify complex patterns become useful. Machine learning fosters several techniques to enhance the detection, prevention, and mitigation of cyberattacks. However, important features are not addressed in the current state-of-the-art security datasets for IoV. For example, in the case of intra-vehicle communications, it is critical to consider the interaction among multiple Electronic Control Units (ECUs). Also, mimicking a realistic IoV environment is not simple since establishing a test environment requires considerable financial investment. Hence, there is a need for a testbed composed of several real ECUs in an IoV environment comprising network traffic. Thereupon, the main goal of this research is to propose a realistic benchmark dataset to foster the development of new cybersecurity solutions for IoV operations. To accomplish this, five attacks were executed against the fully intact inner structure of a 2019 Ford car, complete with all ECUs. However, the vehicle was immobile and incapable of causing any potential harm or injuries. Hence, all attacks were carried out on the vehicle without endangering the car’s driver or passengers. These attacks are classified as spoofing and Denial-of-Service (Dos) and were carried out through the Controller Area Network (CAN) protocol. This effort establishes a baseline complementary to existing contributions and supports researchers in proposing new IoV solutions to strengthen overall security using different techniques (e.g., Machine Learning — ML). The CICIoV2024 dataset has been published on CIC’s dataset page.

Parameter free AEWMA control chart for dispersion in semiconductor manufacturing

The study presents a new parameter free adaptive exponentially weighted moving average (AEWMA) control chart tailored for monitoring process dispersion, utilizing an adaptive approach for determining the smoothing constant. This chart is crafted to adeptly detect shifts within anticipated ranges in process dispersion by dynamically computing the smoothing constant. To assess its effectiveness, the chart’s performance is measured through concise run-length profiles generated from Monte Carlo simulations. A notable aspect is the incorporation of an unbiased estimator in computing the smoothing constant through the suggested function, thereby improving the chart’s capability to identify different levels of increasing and decreasing shifts in process dispersion. The comparison with an established adaptive EWMA-S2 dispersion chart highlights the considerable efficiency of the proposed chart in addressing diverse magnitudes of process dispersion shifts. Additionally, the study includes an application to a real-life dataset, showcasing the practicality and user-friendly nature of the proposed chart in real-world situations.

Enhancing geospatial prediction models with feature engineering from road networks: a graph-driven approach

Traditional geospatial predictive models for property valuation have naturally relied on coordinates as well as ‘hedonic’ (internal and external) features. In particular, location-centric methods such as Geographically Weighted Regression (GWR) and Kriging have focused on intrinsic target characteristics together with distances between individual targets. However, especially in the context of heavily urbanized areas, these approaches might overlook crucial aspects arising from the underlying topological structure that presents itself in such areas. Concretely, in this work, we focus on the structure arising from the road network connecting properties. We introduce a novel though straightforward technique for feature engineering based on graphs constructed on a road network. We then extract relevant features from these and utilize those as inputs for predictive models and assess their performance benefits when used together with a variety of both well-known geospatial models and state-of-art machine learning models. To this end, we present an exhaustive experiment using four different real-life data sets across various regions and exhibiting sizes outperforming many comparative works in the field. Our findings reveal that our feature engineering approach offers significant improvements in predictive performance. Finally, we apply Shapley values as an interpretability technique to confirm the reliability and effectiveness of our approach.

A New Statistical Approach Based on the Access of Electricity Application with Some Modified Control Charts

This article introduces a new probability model based on reflected parameter called the reflected Pareto (RP) distribution. The key properties of the RP model are investigated. A simulation study of the RP model is conducted to evaluate the performances of its estimators. A real-life application is considered to examine the performance of proposed model. The different criteria are discussed numerically as well as graphically to show the flexibility of the RP model. The exponential weighted moving average control charts based on the maximum likelihood and modified maximum likelihood estimators for the shape parameter of the RP distribution are obtained. Detailed simulation results of proposed charts are performed to examine and analyze the performance of these charts with three in-control average run length values and two sample sizes. Finally, the application of the proposed control charts is shown by considering a real-life data set.

Improved extended empirical wavelet transform for accurate multivariate oscillation detection and characterisation in plant-wide industrial control loops

The conventional extended empirical wavelet transform (EEWT) proposed recently is intended to decompose multivariate signals with clear peaks in power spectra without considering the cases where the signals contain high noise levels. Even when dealing with signals with distinct peaks, the EEWT method can still encounter challenges in properly decomposing the signals. However, plant-wide data from industrial control loops, including controllers’ outputs, process variables, and manipulated variables, are commonly corrupted by high levels of noise, which can be introduced at various stages of data acquisition, transmission, and processing within the control system. To address these limitations and ensure the applicability of the EEWT to real-world industrial data with diverse and challenging characteristics, this paper presents an improved version called the improved extended empirical wavelet transform (IEEWT). The IEEWT incorporates noise-reduced power spectra and detrended fluctuation analysis techniques to enhance the decomposition. The proposed method demonstrates accurate multivariate data decomposition for both simulated and real data sets, surpassing the limitations associated with the conventional EEWT.

On classes of consistent tests for the Type I Pareto distribution based on a characterization involving order statistics

We propose new classes of goodness-of-fit tests for the Pareto Type I distribution. These tests are based on a characterization of the Pareto distribution involving order statistics. We derive the limiting null distribution of the tests and also show that the tests are consistent against fixed alternatives. The finite-sample performance of the newly proposed tests are evaluated and compared to some of the existing tests, where it is found that the new tests are competitive in terms of powers. The paper concludes with an application to a real world data set, namely the earnings of the 22 highest paid participants in the inaugural season of LIV golf.

Statistical Analysis of the Inverse Power Zeghdoudi Model: Estimation, Simulation and Modeling to Engineering and Environmental Data

Abstract In this research, we investigate a brand-new two-parameter distribution as an extension of the
power Zeghdoudi distribution (PZD). Using the inverse transformation technique on the PZD, the
produced distribution is called the inverted PZD (IPZD). Its usefulness in producing symmetric and
asymmetric probability density functions makes it the perfect choice for lifetime phenomenon mod-
eling. It is also appropriate for a range of real data since the relevant hazard rate function has one of
the following shapes: increasing, decreasing, reverse j-shape or upside-down shape. Mode, quantiles,
moments, geometric mean, inverse moments, incomplete moments, distribution of order statistics,
Lorenz, Bonferroni, and Zenga curves are a few of the significant characteristics and aspects explored
in our study along with some graphical representations. Twelve effective estimating techniques are
used to determine the distribution parameters of the IPZD. These include the Kolmogorov, least
squares (LS), a maximum product of spacing, Anderson-Darling (AD), maximum likelihood, mini-
mum absolute spacing distance, right-tail AD, minimum absolute spacing-log distance, weighted LS,
left-tailed AD, Cram ́er-von Mises, AD left-tail second-order. A Monte Carlo simulation is used to
examine the effectiveness of the obtained estimates. The visual representation and numerical results
show that the maximum likelihood estimation strategy regularly beats the other methods in terms of
accuracy when estimating the relevant parameters. The usefulness of the recommended distribution
for modelling data is illustrated and displayed visually using two real data sets and comparisons with
other distributions.

The functional distance-based approach: An application on long-term Metropolitan Development

Metropolitan development is a complex phenomenon depending on the joint action of multiple socioeconomic and territorial factors, which are generally quantified through indicators interacting across spatial and temporal scales. A comprehensive understanding of metropolitan development requires a detailed examination of the dependence structure within the development trajectories of both urban and suburban areas. Multidimensionality, non-linearity, and evolutionary components at the base of metropolitan development should be jointly considered when investigating the socioeconomic mechanisms underlying the long-term expansion of cities and their surrounding regions. In this multidimensional perspective, the Functional Data Analysis approach enables the simultaneous investigation of the level and evolution over time of multiple socioeconomic processes through summary functional measures, such as the modified hypograph index and the weighted integrated first derivative. Starting from these tools, a functional distance-based approach is proposed to study the relationship between urban and suburban areas from a twofold perspective: focusing on the entire set of metropolitan development indicators, and considering each dimension individually. This approach contributes to overpass the limitations of standard econometric approaches offering a multivariate measure of change in local systems experiencing complex development trajectories. The proposal is illustrated via a simulation study and a real data set.

Degradation modeling and remaining useful lifetime prediction based on functional variance process

AbstractDynamic fluctuation is a common phenomenon in degradation processes. Hence, how to model it properly has a great impact on the degradation modeling as well as the remaining useful lifetime prediction. To capture the dynamic features and to avoid the risk of the model mis‐specification, a nonparametric degradation model based on functional variance process is proposed in this article. The model is composed of a unit‐specific mean trend and a degradation fluctuation which follows a stochastic process. The mean trend is estimated by the local smoother method, while the stochastic fluctuation is estimated by the functional principal component analysis method. The asymptotic properties of the estimators are proved. Also, the prediction for the remaining useful lifetime is discussed and the estimator is proved to converge in distribution. Moreover, a Bayesian scheme is developed to forecast the remaining useful lifetime for units with incomplete degradation observations. Simulation results show the superiority of the proposed method by comparing it with some existing methods. Finally, two real data sets are analyzed and used to illustrate the application of the method.

DBGSA: A novel data adaptive bregman clustering algorithm

Traditional clustering algorithms such as K-means are highly sensitive to the initial centroid selection and perform poorly on non-convex dataset. To address these problems, a novel data adaptive Bregman clustering algorithm (DBGSA) is proposed in this paper, which combines the universal gravitational algorithm to bring similar points closer in the dataset. In addition, we introduce the Bregman divergence generalized power mean information loss minimization to identify cluster centers and build a hyperparameter identification optimization model, which effectively solves the problems of manual adjustment and uncertainty in the improved dataset. Extensive experiments are conducted on four simulated datasets and six real datasets. The results demonstrate that DBGSA significantly improves the accuracy of various clustering algorithms by an average of 63.8% compared to the contrasted algorithms, making it applicable to diverse datasets.

Something Out of Nothing? The Influence of Double-Zero Studies in Meta-analysis of Adverse Events in Clinical Trials

AbstractIn addition to clinical efficacy, safety is another important outcome to assess in randomized controlled trials. It focuses on the occurrence of adverse events, such as stroke, deaths, and other rare events. Because of the low or very low rates of observing adverse events, meta-analysis is often used to pooled together evidence from dozens or even hundreds of similar clinical trials to strengthen inference. A well-known issue in rare-event meta-analysis is that some or even majority of the available studies may observe zero events in both the treatment and control groups. The influence of these so-called double-zero studies has been researched in the literature, which nevertheless focuses on reaching a dichotomous conclusion—whether or not double-zero studies should be included in the analysis. It has not been addressed when and how they contribute to inference, especially for the odds ratio. This paper fills this gap using comparative analysis of real and simulated data sets. We find that a double-zero study contributes to the odds ratio inference through its sample sizes in the two arms. When a double-zero study has an unequal allocation of patients in its two arms, it may contain non-ignorable information. Exclusion of these studies, if taking a significant proportion of the study cohort, may result in inflated type I error, deteriorated testing power, and increased estimation bias.

Informed Adaptive Sensing.

For many inverse problems, the data on which the solution is based is acquired sequentially. We present an approach to the solution of such inverse problems where a sensor can be directed (or otherwise reconfigured on the fly) to acquire a particular measurement. An example problem is magnetic resonance image reconstruction. We use an estimate of mutual information derived from an empirical conditional distribution provided by a generative model to guide our measurement acquisition given measurements acquired so far. The conditionally generated data is a set of samples which are representative of the plausible solutions that satisfy the acquired measurements. We present experiments on toy and real world data sets. We focus on image data but we demonstrate that the method is applicable to a broader class of problems. We also show how a learned model such as a deep neural network can be leveraged to allow generalisation to unseen data. Our informed adaptive sensing method outperforms random sampling, variance based sampling, sparsity based methods, and compressed sensing.

A New Sub-Class Linear Discriminant for Miniature Spectrometer Based Food Analysis

The well-known and extensively studied Linear Discriminant Analysis (LDA) can have its performance lowered in scenarios where data is not homoscedastic or not Gaussian. That is, the classical assumptions when LDA models are built are not applicable, and consequently LDA projections would not be able to extract the needed features to explain the intrinsic structure of data and for classes to be separated. As with many real word data sets, data obtained using miniature spectrometers can suffer from such drawbacks which would limit the deployment of such technology needed for food analysis. The solution presented in the paper is to divide classes into subclasses and to use means of sub classes, classes, and data in the suggested between classes scatter metric. Further, samples belonging to the same subclass are used to build a measure of within subclass scatterness. Such a solution solves the shortcoming of the classical LDA. The obtained results when using the proposed solution on food data and on general machine learning datasets show that the work in this paper compares well to and is very competitive with similar sub-class LDA algorithms in the literature. An extension to a Hilbert space is also presented; and the kernel version of the presented solution can be fused with its linear counter parts to yield improved classification rates.